{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# <div style='color:#fd7949;font-weight:800;'>超参数调优库optuna介绍</div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "An open source hyperparameter optimization framework to automate hyperparameter search\n",
    "\n",
    "\n",
    "\n",
    "Optuna is framework agnostic. You can use it with any machine learning or deep learning framework."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 二、Optuna简单例子"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A simple optimization problem:\n",
    "\n",
    "- Define objective function to be optimized. Let's minimize (x - 2)^2\n",
    "- Suggest hyperparameter values using trial object. Here, a float value of x is suggested from -10 to 10\n",
    "- Create a study object and invoke the optimize method over 100 trials"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[32m[I 2022-03-09 06:43:56,040]\u001b[0m A new study created in memory with name: no-name-37d9e384-0645-43c3-bf5d-9ea97be2d8a1\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,042]\u001b[0m Trial 0 finished with value: 3.6336410559517947 and parameters: {'x': 3.9062111782149938}. Best is trial 0 with value: 3.6336410559517947.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,043]\u001b[0m Trial 1 finished with value: 25.61383483826753 and parameters: {'x': 7.061011246605517}. Best is trial 0 with value: 3.6336410559517947.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,044]\u001b[0m Trial 2 finished with value: 50.89324884383307 and parameters: {'x': -5.133950437438788}. Best is trial 0 with value: 3.6336410559517947.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,046]\u001b[0m Trial 3 finished with value: 0.12694501165434258 and parameters: {'x': 1.643706565238226}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,048]\u001b[0m Trial 4 finished with value: 7.287041397548649 and parameters: {'x': 4.699452055056479}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,048]\u001b[0m Trial 5 finished with value: 0.23269705938855648 and parameters: {'x': 2.4823868358367136}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,050]\u001b[0m Trial 6 finished with value: 49.18661548412601 and parameters: {'x': 9.013317010097719}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,050]\u001b[0m Trial 7 finished with value: 10.688539315851388 and parameters: {'x': -1.269333160730394}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,051]\u001b[0m Trial 8 finished with value: 14.653137060600795 and parameters: {'x': 5.8279416218903854}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,052]\u001b[0m Trial 9 finished with value: 120.54665697833781 and parameters: {'x': -8.979374161505646}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,057]\u001b[0m Trial 10 finished with value: 10.166755451404988 and parameters: {'x': -1.1885350008122835}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,063]\u001b[0m Trial 11 finished with value: 0.8247279498088238 and parameters: {'x': 1.091854664820204}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,069]\u001b[0m Trial 12 finished with value: 0.15478721388888955 and parameters: {'x': 1.6065699377412936}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,073]\u001b[0m Trial 13 finished with value: 33.64646298798943 and parameters: {'x': -3.8005571273791827}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,080]\u001b[0m Trial 14 finished with value: 1.8331432356702748 and parameters: {'x': 0.6460637992614737}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,085]\u001b[0m Trial 15 finished with value: 30.93079541221247 and parameters: {'x': -3.5615461350430664}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,089]\u001b[0m Trial 16 finished with value: 0.47111961228163435 and parameters: {'x': 2.686381535504587}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,094]\u001b[0m Trial 17 finished with value: 57.381250761040334 and parameters: {'x': 9.575041304246488}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,101]\u001b[0m Trial 18 finished with value: 93.85888004692498 and parameters: {'x': -7.688079275425288}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,106]\u001b[0m Trial 19 finished with value: 5.196627983587673 and parameters: {'x': -0.27961136678769694}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,111]\u001b[0m Trial 20 finished with value: 18.041195792200806 and parameters: {'x': -2.2474928831253864}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,118]\u001b[0m Trial 21 finished with value: 0.33385960395973435 and parameters: {'x': 2.5778058531719235}. Best is trial 3 with value: 0.12694501165434258.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,123]\u001b[0m Trial 22 finished with value: 0.012448139481939574 and parameters: {'x': 1.888428769470174}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,131]\u001b[0m Trial 23 finished with value: 23.082684838308886 and parameters: {'x': 6.804444279863061}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,137]\u001b[0m Trial 24 finished with value: 0.2115114504522738 and parameters: {'x': 1.5400962595800358}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,142]\u001b[0m Trial 25 finished with value: 4.321133004241607 and parameters: {'x': 4.078733509674005}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,147]\u001b[0m Trial 26 finished with value: 4.348776301324235 and parameters: {'x': -0.0853719815237366}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,152]\u001b[0m Trial 27 finished with value: 13.637052447946678 and parameters: {'x': 5.692837993731471}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,157]\u001b[0m Trial 28 finished with value: 61.92120225449036 and parameters: {'x': -5.869002621329488}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,163]\u001b[0m Trial 29 finished with value: 1.4107601640433305 and parameters: {'x': 3.187754252378551}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,169]\u001b[0m Trial 30 finished with value: 25.341180727144582 and parameters: {'x': -3.034002456012967}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,175]\u001b[0m Trial 31 finished with value: 0.6648822517698565 and parameters: {'x': 1.1845968777531837}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,181]\u001b[0m Trial 32 finished with value: 0.10024947277031887 and parameters: {'x': 2.316621971395415}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,186]\u001b[0m Trial 33 finished with value: 7.732368299055246 and parameters: {'x': 4.7807136312564165}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,193]\u001b[0m Trial 34 finished with value: 3.023193662671384 and parameters: {'x': 3.7387333500774016}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,201]\u001b[0m Trial 35 finished with value: 0.040312052659596316 and parameters: {'x': 1.79922138395836}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,206]\u001b[0m Trial 36 finished with value: 34.97072523007271 and parameters: {'x': 7.913605095884634}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,212]\u001b[0m Trial 37 finished with value: 8.560331892638693 and parameters: {'x': -0.9258044864000556}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,218]\u001b[0m Trial 38 finished with value: 7.602943581247006 and parameters: {'x': 4.757343573305112}. Best is trial 22 with value: 0.012448139481939574.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,225]\u001b[0m Trial 39 finished with value: 0.002692314683340682 and parameters: {'x': 2.051887519533513}. Best is trial 39 with value: 0.002692314683340682.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,231]\u001b[0m Trial 40 finished with value: 0.10302430432875971 and parameters: {'x': 2.320973993228049}. Best is trial 39 with value: 0.002692314683340682.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,237]\u001b[0m Trial 41 finished with value: 0.4134874952153983 and parameters: {'x': 2.6430299333743323}. Best is trial 39 with value: 0.002692314683340682.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,242]\u001b[0m Trial 42 finished with value: 2.481574423615193 and parameters: {'x': 0.42469862451174345}. Best is trial 39 with value: 0.002692314683340682.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,248]\u001b[0m Trial 43 finished with value: 2.826699878036219 and parameters: {'x': 3.681279238566937}. Best is trial 39 with value: 0.002692314683340682.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,254]\u001b[0m Trial 44 finished with value: 2.836432737716509e-07 and parameters: {'x': 2.000532581706193}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,258]\u001b[0m Trial 45 finished with value: 13.066194907500483 and parameters: {'x': -1.6147192017500454}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,267]\u001b[0m Trial 46 finished with value: 12.303923316401935 and parameters: {'x': 5.507694872191983}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,273]\u001b[0m Trial 47 finished with value: 0.03204209460349564 and parameters: {'x': 1.8209969424744492}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,280]\u001b[0m Trial 48 finished with value: 4.590345260908871 and parameters: {'x': -0.1425091040434041}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,286]\u001b[0m Trial 49 finished with value: 1.3970727795861342 and parameters: {'x': 0.8180216670403242}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,293]\u001b[0m Trial 50 finished with value: 0.04239670187613436 and parameters: {'x': 1.7940954058887117}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,302]\u001b[0m Trial 51 finished with value: 0.12626382051394305 and parameters: {'x': 1.6446637922840637}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,309]\u001b[0m Trial 52 finished with value: 0.05447126933722297 and parameters: {'x': 1.7666091918321911}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,315]\u001b[0m Trial 53 finished with value: 1.8223826647447032 and parameters: {'x': 3.349956541798551}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,321]\u001b[0m Trial 54 finished with value: 5.854181213966002 and parameters: {'x': 4.41954152970475}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,327]\u001b[0m Trial 55 finished with value: 5.929648105569726 and parameters: {'x': -0.43508687844391414}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,334]\u001b[0m Trial 56 finished with value: 2.331149830147317 and parameters: {'x': 0.4731896548204433}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,344]\u001b[0m Trial 57 finished with value: 13.752335553121203 and parameters: {'x': -1.7084141560943813}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,356]\u001b[0m Trial 58 finished with value: 1.0506580686060327 and parameters: {'x': 3.0250161309004033}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,364]\u001b[0m Trial 59 finished with value: 0.0005497075567123622 and parameters: {'x': 2.023445843058256}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,369]\u001b[0m Trial 60 finished with value: 18.463834244482438 and parameters: {'x': 6.29695639313252}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,374]\u001b[0m Trial 61 finished with value: 0.2485894914813465 and parameters: {'x': 1.5014125036853145}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,381]\u001b[0m Trial 62 finished with value: 1.589130810791373 and parameters: {'x': 0.7393926817635188}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,388]\u001b[0m Trial 63 finished with value: 0.013968532713235038 and parameters: {'x': 2.1181885473014836}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,394]\u001b[0m Trial 64 finished with value: 0.008688532616801572 and parameters: {'x': 2.093212298634899}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,401]\u001b[0m Trial 65 finished with value: 7.2279574128720405 and parameters: {'x': -0.6884860819561704}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,406]\u001b[0m Trial 66 finished with value: 0.07723724367299219 and parameters: {'x': 2.277915893163727}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,413]\u001b[0m Trial 67 finished with value: 1.427241822797931 and parameters: {'x': 3.1946722658528284}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,418]\u001b[0m Trial 68 finished with value: 9.585037401811924 and parameters: {'x': 5.095971156489014}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,425]\u001b[0m Trial 69 finished with value: 3.7686698006170127 and parameters: {'x': 3.941306209905334}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,431]\u001b[0m Trial 70 finished with value: 3.453258612680288 and parameters: {'x': 0.14170545588696082}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,437]\u001b[0m Trial 71 finished with value: 0.6904423586374024 and parameters: {'x': 1.1690713877610168}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,443]\u001b[0m Trial 72 finished with value: 0.14545851285551833 and parameters: {'x': 2.3813902369693256}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,449]\u001b[0m Trial 73 finished with value: 0.018063054877130787 and parameters: {'x': 2.1343988648654846}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,454]\u001b[0m Trial 74 finished with value: 0.9875349693066289 and parameters: {'x': 2.993747940529503}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,459]\u001b[0m Trial 75 finished with value: 0.8331050367038206 and parameters: {'x': 1.0872541226037662}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,466]\u001b[0m Trial 76 finished with value: 0.07323652416194223 and parameters: {'x': 2.270622475345161}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,471]\u001b[0m Trial 77 finished with value: 4.480429882838673 and parameters: {'x': 4.116702596691059}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,476]\u001b[0m Trial 78 finished with value: 2.169216800325956 and parameters: {'x': 3.472826126983751}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,483]\u001b[0m Trial 79 finished with value: 0.5420308384051391 and parameters: {'x': 2.7362274366017196}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,489]\u001b[0m Trial 80 finished with value: 3.2445227148456786 and parameters: {'x': 0.1987441284354745}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,495]\u001b[0m Trial 81 finished with value: 0.0048292640340044015 and parameters: {'x': 1.9305070936425566}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,502]\u001b[0m Trial 82 finished with value: 0.0035433791838588064 and parameters: {'x': 2.0595262898546416}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,508]\u001b[0m Trial 83 finished with value: 0.7159779930182484 and parameters: {'x': 1.153845171958318}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,514]\u001b[0m Trial 84 finished with value: 2.310813086842958e-05 and parameters: {'x': 2.0048070917266503}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,520]\u001b[0m Trial 85 finished with value: 0.6387846869853634 and parameters: {'x': 2.7992400684308585}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,526]\u001b[0m Trial 86 finished with value: 2.066557088901937 and parameters: {'x': 0.562447535252387}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,532]\u001b[0m Trial 87 finished with value: 0.3313218778658821 and parameters: {'x': 1.4243943382263495}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,537]\u001b[0m Trial 88 finished with value: 139.21981435770766 and parameters: {'x': -9.799144645172703}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,543]\u001b[0m Trial 89 finished with value: 0.0018284776587347343 and parameters: {'x': 1.9572392977287003}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,549]\u001b[0m Trial 90 finished with value: 2.733594692586016 and parameters: {'x': 3.653358609795835}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,555]\u001b[0m Trial 91 finished with value: 0.0030712691421836786 and parameters: {'x': 1.9445809676899382}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,560]\u001b[0m Trial 92 finished with value: 1.1146828685923005 and parameters: {'x': 0.9442145726558353}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,567]\u001b[0m Trial 93 finished with value: 0.014762855745960306 and parameters: {'x': 1.8784975072438417}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,574]\u001b[0m Trial 94 finished with value: 0.878416914515959 and parameters: {'x': 2.9372389847397296}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,581]\u001b[0m Trial 95 finished with value: 4.5809126670413685 and parameters: {'x': 4.140306675932533}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,588]\u001b[0m Trial 96 finished with value: 0.3308429358828501 and parameters: {'x': 2.575189478244213}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,593]\u001b[0m Trial 97 finished with value: 0.19827105234959616 and parameters: {'x': 1.5547236225111463}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,600]\u001b[0m Trial 98 finished with value: 4.69900365812418 and parameters: {'x': -0.16771853756989863}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n",
      "\u001b[32m[I 2022-03-09 06:43:56,608]\u001b[0m Trial 99 finished with value: 1.9424143462488745 and parameters: {'x': 3.3937052580258404}. Best is trial 44 with value: 2.836432737716509e-07.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "application/vnd.plotly.v1+json": {
       "config": {
        "plotlyServerURL": "https://plot.ly"
       },
       "data": [
        {
         "mode": "markers",
         "name": "Objective Value",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          3.6336410559517947,
          25.61383483826753,
          50.89324884383307,
          0.12694501165434258,
          7.287041397548649,
          0.23269705938855648,
          49.18661548412601,
          10.688539315851388,
          14.653137060600795,
          120.54665697833781,
          10.166755451404988,
          0.8247279498088238,
          0.15478721388888955,
          33.64646298798943,
          1.8331432356702748,
          30.93079541221247,
          0.47111961228163435,
          57.381250761040334,
          93.85888004692498,
          5.196627983587673,
          18.041195792200806,
          0.33385960395973435,
          0.012448139481939574,
          23.082684838308886,
          0.2115114504522738,
          4.321133004241607,
          4.348776301324235,
          13.637052447946678,
          61.92120225449036,
          1.4107601640433305,
          25.341180727144582,
          0.6648822517698565,
          0.10024947277031887,
          7.732368299055246,
          3.023193662671384,
          0.040312052659596316,
          34.97072523007271,
          8.560331892638693,
          7.602943581247006,
          0.002692314683340682,
          0.10302430432875971,
          0.4134874952153983,
          2.481574423615193,
          2.826699878036219,
          2.836432737716509e-7,
          13.066194907500483,
          12.303923316401935,
          0.03204209460349564,
          4.590345260908871,
          1.3970727795861342,
          0.04239670187613436,
          0.12626382051394305,
          0.05447126933722297,
          1.8223826647447032,
          5.854181213966002,
          5.929648105569726,
          2.331149830147317,
          13.752335553121203,
          1.0506580686060327,
          0.0005497075567123622,
          18.463834244482438,
          0.2485894914813465,
          1.589130810791373,
          0.013968532713235038,
          0.008688532616801572,
          7.2279574128720405,
          0.07723724367299219,
          1.427241822797931,
          9.585037401811924,
          3.7686698006170127,
          3.453258612680288,
          0.6904423586374024,
          0.14545851285551833,
          0.018063054877130787,
          0.9875349693066289,
          0.8331050367038206,
          0.07323652416194223,
          4.480429882838673,
          2.169216800325956,
          0.5420308384051391,
          3.2445227148456786,
          0.0048292640340044015,
          0.0035433791838588064,
          0.7159779930182484,
          0.00002310813086842958,
          0.6387846869853634,
          2.066557088901937,
          0.3313218778658821,
          139.21981435770766,
          0.0018284776587347343,
          2.733594692586016,
          0.0030712691421836786,
          1.1146828685923005,
          0.014762855745960306,
          0.878416914515959,
          4.5809126670413685,
          0.3308429358828501,
          0.19827105234959616,
          4.69900365812418,
          1.9424143462488745
         ]
        },
        {
         "name": "Best Value",
         "type": "scatter",
         "x": [
          0,
          1,
          2,
          3,
          4,
          5,
          6,
          7,
          8,
          9,
          10,
          11,
          12,
          13,
          14,
          15,
          16,
          17,
          18,
          19,
          20,
          21,
          22,
          23,
          24,
          25,
          26,
          27,
          28,
          29,
          30,
          31,
          32,
          33,
          34,
          35,
          36,
          37,
          38,
          39,
          40,
          41,
          42,
          43,
          44,
          45,
          46,
          47,
          48,
          49,
          50,
          51,
          52,
          53,
          54,
          55,
          56,
          57,
          58,
          59,
          60,
          61,
          62,
          63,
          64,
          65,
          66,
          67,
          68,
          69,
          70,
          71,
          72,
          73,
          74,
          75,
          76,
          77,
          78,
          79,
          80,
          81,
          82,
          83,
          84,
          85,
          86,
          87,
          88,
          89,
          90,
          91,
          92,
          93,
          94,
          95,
          96,
          97,
          98,
          99
         ],
         "y": [
          3.6336410559517947,
          3.6336410559517947,
          3.6336410559517947,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.12694501165434258,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.012448139481939574,
          0.002692314683340682,
          0.002692314683340682,
          0.002692314683340682,
          0.002692314683340682,
          0.002692314683340682,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7,
          2.836432737716509e-7
         ]
        }
       ],
       "layout": {
        "template": {
         "data": {
          "bar": [
           {
            "error_x": {
             "color": "#2a3f5f"
            },
            "error_y": {
             "color": "#2a3f5f"
            },
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "bar"
           }
          ],
          "barpolar": [
           {
            "marker": {
             "line": {
              "color": "#E5ECF6",
              "width": 0.5
             },
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "barpolar"
           }
          ],
          "carpet": [
           {
            "aaxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "baxis": {
             "endlinecolor": "#2a3f5f",
             "gridcolor": "white",
             "linecolor": "white",
             "minorgridcolor": "white",
             "startlinecolor": "#2a3f5f"
            },
            "type": "carpet"
           }
          ],
          "choropleth": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "choropleth"
           }
          ],
          "contour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "contour"
           }
          ],
          "contourcarpet": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "contourcarpet"
           }
          ],
          "heatmap": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmap"
           }
          ],
          "heatmapgl": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "heatmapgl"
           }
          ],
          "histogram": [
           {
            "marker": {
             "pattern": {
              "fillmode": "overlay",
              "size": 10,
              "solidity": 0.2
             }
            },
            "type": "histogram"
           }
          ],
          "histogram2d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2d"
           }
          ],
          "histogram2dcontour": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "histogram2dcontour"
           }
          ],
          "mesh3d": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "type": "mesh3d"
           }
          ],
          "parcoords": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "parcoords"
           }
          ],
          "pie": [
           {
            "automargin": true,
            "type": "pie"
           }
          ],
          "scatter": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter"
           }
          ],
          "scatter3d": [
           {
            "line": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatter3d"
           }
          ],
          "scattercarpet": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattercarpet"
           }
          ],
          "scattergeo": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergeo"
           }
          ],
          "scattergl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattergl"
           }
          ],
          "scattermapbox": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scattermapbox"
           }
          ],
          "scatterpolar": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolar"
           }
          ],
          "scatterpolargl": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterpolargl"
           }
          ],
          "scatterternary": [
           {
            "marker": {
             "colorbar": {
              "outlinewidth": 0,
              "ticks": ""
             }
            },
            "type": "scatterternary"
           }
          ],
          "surface": [
           {
            "colorbar": {
             "outlinewidth": 0,
             "ticks": ""
            },
            "colorscale": [
             [
              0,
              "#0d0887"
             ],
             [
              0.1111111111111111,
              "#46039f"
             ],
             [
              0.2222222222222222,
              "#7201a8"
             ],
             [
              0.3333333333333333,
              "#9c179e"
             ],
             [
              0.4444444444444444,
              "#bd3786"
             ],
             [
              0.5555555555555556,
              "#d8576b"
             ],
             [
              0.6666666666666666,
              "#ed7953"
             ],
             [
              0.7777777777777778,
              "#fb9f3a"
             ],
             [
              0.8888888888888888,
              "#fdca26"
             ],
             [
              1,
              "#f0f921"
             ]
            ],
            "type": "surface"
           }
          ],
          "table": [
           {
            "cells": {
             "fill": {
              "color": "#EBF0F8"
             },
             "line": {
              "color": "white"
             }
            },
            "header": {
             "fill": {
              "color": "#C8D4E3"
             },
             "line": {
              "color": "white"
             }
            },
            "type": "table"
           }
          ]
         },
         "layout": {
          "annotationdefaults": {
           "arrowcolor": "#2a3f5f",
           "arrowhead": 0,
           "arrowwidth": 1
          },
          "autotypenumbers": "strict",
          "coloraxis": {
           "colorbar": {
            "outlinewidth": 0,
            "ticks": ""
           }
          },
          "colorscale": {
           "diverging": [
            [
             0,
             "#8e0152"
            ],
            [
             0.1,
             "#c51b7d"
            ],
            [
             0.2,
             "#de77ae"
            ],
            [
             0.3,
             "#f1b6da"
            ],
            [
             0.4,
             "#fde0ef"
            ],
            [
             0.5,
             "#f7f7f7"
            ],
            [
             0.6,
             "#e6f5d0"
            ],
            [
             0.7,
             "#b8e186"
            ],
            [
             0.8,
             "#7fbc41"
            ],
            [
             0.9,
             "#4d9221"
            ],
            [
             1,
             "#276419"
            ]
           ],
           "sequential": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ],
           "sequentialminus": [
            [
             0,
             "#0d0887"
            ],
            [
             0.1111111111111111,
             "#46039f"
            ],
            [
             0.2222222222222222,
             "#7201a8"
            ],
            [
             0.3333333333333333,
             "#9c179e"
            ],
            [
             0.4444444444444444,
             "#bd3786"
            ],
            [
             0.5555555555555556,
             "#d8576b"
            ],
            [
             0.6666666666666666,
             "#ed7953"
            ],
            [
             0.7777777777777778,
             "#fb9f3a"
            ],
            [
             0.8888888888888888,
             "#fdca26"
            ],
            [
             1,
             "#f0f921"
            ]
           ]
          },
          "colorway": [
           "#636efa",
           "#EF553B",
           "#00cc96",
           "#ab63fa",
           "#FFA15A",
           "#19d3f3",
           "#FF6692",
           "#B6E880",
           "#FF97FF",
           "#FECB52"
          ],
          "font": {
           "color": "#2a3f5f"
          },
          "geo": {
           "bgcolor": "white",
           "lakecolor": "white",
           "landcolor": "#E5ECF6",
           "showlakes": true,
           "showland": true,
           "subunitcolor": "white"
          },
          "hoverlabel": {
           "align": "left"
          },
          "hovermode": "closest",
          "mapbox": {
           "style": "light"
          },
          "paper_bgcolor": "white",
          "plot_bgcolor": "#E5ECF6",
          "polar": {
           "angularaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "radialaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "scene": {
           "xaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "yaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           },
           "zaxis": {
            "backgroundcolor": "#E5ECF6",
            "gridcolor": "white",
            "gridwidth": 2,
            "linecolor": "white",
            "showbackground": true,
            "ticks": "",
            "zerolinecolor": "white"
           }
          },
          "shapedefaults": {
           "line": {
            "color": "#2a3f5f"
           }
          },
          "ternary": {
           "aaxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "baxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           },
           "bgcolor": "#E5ECF6",
           "caxis": {
            "gridcolor": "white",
            "linecolor": "white",
            "ticks": ""
           }
          },
          "title": {
           "x": 0.05
          },
          "xaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          },
          "yaxis": {
           "automargin": true,
           "gridcolor": "white",
           "linecolor": "white",
           "ticks": "",
           "title": {
            "standoff": 15
           },
           "zerolinecolor": "white",
           "zerolinewidth": 2
          }
         }
        },
        "title": {
         "text": "Optimization History Plot"
        },
        "xaxis": {
         "title": {
          "text": "#Trials"
         }
        },
        "yaxis": {
         "title": {
          "text": "Objective Value"
         }
        }
       }
      }
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import optuna\n",
    "\n",
    "def objective(trial):\n",
    "    x = trial.suggest_float('x', -10, 10)\n",
    "    return (x - 2) ** 2\n",
    "\n",
    "study = optuna.create_study()\n",
    "study.optimize(objective, n_trials=100)\n",
    "\n",
    "study.best_params  # E.g. {'x': 2.002108042}\n",
    "\n",
    "optuna.visualization.plot_optimization_history(study)\n"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "9465aae7e0ab1403d672807d1a0963d86dbda2f584fbe3054c36cf78311c6c77"
  },
  "kernelspec": {
   "display_name": "Python 3.8.11 ('pytorch')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.11"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
